probability theory Probability theory is the branch of mathematics concerned with probability. Although there are several different probability interpretations, probability theory treats the concept in a rigorous mathematical manner by expressing it through a set o ...

, the law of total covariance, covariance decomposition formula, or conditional covariance formula states that if ''X'', ''Y'', and ''Z'' are

random variable A random variable (also called random quantity, aleatory variable, or stochastic variable) is a mathematical formalization of a quantity or object which depends on random events. It is a mapping or a function from possible outcomes (e.g., the po ...

s on the same

probability space In probability theory, a probability space or a probability triple (\Omega, \mathcal, P) is a mathematical construct that provides a formal model of a random process or "experiment". For example, one can define a probability space which models t ...

, and the

covariance In probability theory and statistics, covariance is a measure of the joint variability of two random variables. If the greater values of one variable mainly correspond with the greater values of the other variable, and the same holds for the ...

of ''X'' and ''Y'' is finite, then :

\operatorname(X,Y)=\operatorname(\operatorname(X,Y \mid Z))+\operatorname(\operatorname(X\mid Z),\operatorname(Y\mid Z)).

The nomenclature in this article's title parallels the phrase ''

law of total variance In probability theory, the law of total variance or variance decomposition formula or conditional variance formulas or law of iterated variances also known as Eve's law, states that if X and Y are random variables on the same probability space, and ...

''. Some writers on probability call this the "conditional covariance formula"Sheldon M. Ross, ''A First Course in Probability'', sixth edition, Prentice Hall, 2002, page 392. or use other names. Note: The

conditional expected value In probability theory, the conditional expectation, conditional expected value, or conditional mean of a random variable is its expected value – the value it would take “on average” over an arbitrarily large number of occurrences – given ...

s E( ''X'' , ''Z'' ) and E( ''Y'' , ''Z'' ) are random variables whose values depend on the value of ''Z''. Note that the conditional expected value of ''X'' given the ''event'' ''Z'' = ''z'' is a function of ''z''. If we write E( ''X'' , ''Z'' = ''z'') = ''g''(''z'') then the random variable E( ''X'' , ''Z'' ) is ''g''(''Z''). Similar comments apply to the conditional covariance.

Proof

The law of total covariance can be proved using the

law of total expectation The proposition in probability theory known as the law of total expectation, the law of iterated expectations (LIE), Adam's law, the tower rule, and the smoothing theorem, among other names, states that if X is a random variable whose expected v ...

: First, :

\operatorname(X,Y) = \operatorname Y - \operatorname operatorname /math>

from a simple standard identity on covariances.  Then we apply the law of total expectation by conditioning on the random variable ''Z'':

:: = \operatorname\big operatorname[XY\mid_Z big.html" ;"title="Y\mid_Z.html" ;"title="operatorname[XY\mid Z">operatorname[XY\mid Zbig">Y\mid_Z.html" ;"title="operatorname[XY\mid Z">operatorname[XY\mid Zbig- \operatorname\big[\operatorname \mid Z big]\operatorname\big[\operatorname \mid Z big] Now we rewrite the term inside the first expectation using the definition of covariance:

:: = \operatorname\!\big \mid_Z operatorname[Y\mid_Z.html" ;"title="operatorname(X,Y\mid Z) + \operatorname \mid Z operatorname[Y\mid Z">operatorname(X,Y\mid Z) + \operatorname \mid Z operatorname[Y\mid Zbig] - \operatorname\big[\operatorname \mid Z big]\operatorname\big[\operatorname \mid Z big] Since expectation of a sum is the sum of expectations, we can regroup the terms:

:: = \operatorname\!\left operatorname(X,Y\mid Z) + \operatorname operatorname[X\mid_Z \operatorname[Y\mid_Z.html" ;"title="\mid_Z.html" ;"title="operatorname[X\mid Z">operatorname[X\mid Z\operatorname[Y\mid Z">\mid_Z.html" ;"title="operatorname[X\mid Z">operatorname[X\mid Z\operatorname[Y\mid Zright] - \operatorname[\operatorname[X\mid Z\operatorname[\operatorname[Y\mid Z Finally, we recognize the final two terms as the covariance of the conditional expectations E[''X'' ,  ''Z''] and E'Y'' ,  ''Z'' :: = \operatorname\big operatorname(X,Y \mid Z)\big \operatorname\big(\operatorname \mid Z \operatorname \mid Z big)

Notes and references

{{DEFAULTSORT:Law Of Total Covariance Algebra of random variables Covariance and correlation Articles containing proofs Theory of probability distributions Theorems in statistics Statistical laws

Proof

See also

Notes and references